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Claim 1 57. (Amended) A system for automatically videoing the movements of one or more 

participants or objects as they move about within the entire performance ansa of a sporting, 
theatre or concert event, throughout the entire time duration of the event , comprising: 

a first set of multiple stationary overhead cameras , not capable of pan, tilt or zoom 
movement sufficient to form a contiguous field-of-view of said entire performance 
area, for generating a first video stream of images continuously throughout said entire 
time duration ; 

a first algorithm operated on a computer system exclusively and solely responsive to 
said first stream of video images for continuously and simultaneously analyzing the 
images from all cameras in said first set of multiple stationary cameras in order to 
detect the presence of each participant or object within each and every camera* s view, 
to determine for each detected participant or object its relative two-dimensional 
centroid coordinates within that view, and to use this determined information from all 
said first set cameras for updating a real-time tracking database of at least the two- 
dimensional centroid location of each participant or object , relative to said entire 
performance area, where said real-time tracking database includes ail current and past 
said centroid locations determined throughout said entire time of said event, and 

a second algorithm operated on a computer system exclusively and solely responsive 
to said real-time tracking database for automatically and individually adjusting the 
current view of each camera in a second set of multiple stationary cameras, capable of 
pan, tilt and zoom movement distinct from said first set of stationary cameras^ wherein 
each said second set camera is automatically directed without operator intervention to 
maintain an independent view of the participants or objects, where said second set of 
cameras outputs a second stream of video images for viewing and recording and 
wherein said second stream of video images is not used to either determine any 
participant's or object's two-dimensional centroid locations or to otherwise update 
said real-time tracking database. 

Claim i 58. (Amended) The system of claim 1 57 wherein the contiguous field-of-view formed by 
said first set of stationary overhead cameras is substantially parallel to the ground surface 
within said entire performance area. 
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Claim 1 59. (Amended) The system of claim 1 57 for further providing a three dimensional model of 
the movements of one or more participants or objects, further comprising: 

a third algorithm operated on a computer system responsive to said real-time tracking 
database and said first and said second streams of video images for determining the 
ongoing relative three-dimensional coordinates of one or more specific, non-centroid 
locations on each participant or object and for updating said real-time tracking 
database to include the additional threeHJimensional coordinates of all detected 
locations matched with the respective participants or objects. 

Claim 1 60. (Amended) The system of claim 1 59 further comprising: 

markers adhered onto one or more locations on each participant or object to be 
tracked, where said markers are detectable by said first or said second set of stationary 
cameras, and where said first or said third algorithms now determine the locations of 
said markers and update said real-time tracking database with the. related ongoing 
three-dimensional coordinates of each marker for forming the three dimensional 
model of each participant's movements . 

Claim 1 6 1 . (Amended) The system of claim 1 60 wherein said markers are substantially transparent to 
the participants, further comprising: 

said markers adhered onto participants that reflect, retroreflect or fluoresce primarily 
non-visible energy and are therefore substantially visibly transparent, and 

one or more energy sources emitting non-visible energy that is reflected or 
retroreflected off of said markers, or emitting energy that is fluoresced in the non- 
visible spectrum by said markers and is detectable by said first or said second set of 
stationary cameras. 

Claim 162. (Amended) The system of claim J 61 wherein the participants are additionally identified, 
further comprising: 

at least one uniquely encoded marker adhered onto an upper surface of each 
participant to be identified, and 

a forth algorithm operated on a computer system for locating and recognizing said 
encoded markers within said first stream of video images and for updating said real- 
time tracking database with each participant's identity matched to their coordinates. 
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Claim 163. (Amended) The system of claim 162 wherein said uniquely encoded markers are 
substantially transparent to the participants, further comprising: 

said uniquely encoded markers that reflect, retro reflect or fluoresce primarily non- 
visible energy and are therefore substantially visibly transparent, and 

one or more energy sources emitting non-visible energy that is reflected or 
retroreflected off of said encoded markers, or emitting energy that is fluoresced in the 
non-visible spectrum by said encoded markers and is detectable by said first set of 
stationary overhead cameras. 

Claim 164. (Amended) The system of claim 157 wherein the participants are additionally identified, 
further comprising: 

at least one uniquely encoded marker adhered onto an upper surface of each 
participant to be identified, and 

a third algorithm operated on a computer system for locating and recognizing said 
encoded markers within said first stream of video images and for updating said real- 
time tracking database with each participant's identity matched to their coordinates. 

Claim 165. (Amended) The system of claim 1 64 wherein said uniquely encoded markers are 
substantially transparent to the participants, further comprising: 

said uniquely encoded markers that reflect, retro reflect or fluoresce primarily non- 
visible energy and are therefore substantially visibly transparent, and 

one or more energy sources emitting non -visible energy that is reflected or 
retroreflected off of said markers, or emitting energy that is fluoresced in the non- 
visible spectrum by said markers and is detectable by said first set of stationary 
overhead cameras. 

Claim 1 66. (Amended) The system of claim 1 57 for videoing the movements of two or more 

participants, wherein said second set of stationary camera s, capable of pan, tilt and zoom 
movement comprises at least two cameras and wherein said second set is additionally 
directed to automatically reassign any one or more of said second set cameras currently 
following any one or more participants to instead follow a different one or more 
participants based upon which camera views are currently blocked by one or more 


Page 3 of 7 


PACE 6/18 * RCVD AT 9/9/2008 12:05:05 PM [Eastern Daylight Time) • SVR:USPTO-EFXRF-6/14 * DNIS:2738300 * CSID:Fax 


* DURATION (mm-ss): 06-32 


FmiSmartek, Inc. TorApp SN 10/006,444 312 Amendment (15712738300) 


12:05 09/09/08GMT-04 Pg 07-18 


312 Amended Claims - final 

participants in front of another, or upon which camera views are best situated to capture the 
programmatically desired view of any given participant or game object. 

Claim 167. (Amended) A method for automatically videoing the movements of one or more 

participants or objects as they move about within the entire performance area of a sporting, 
theatre or concert event, throughout the entire time duration of the event , comprising the 
steps of: 

capturing a first stream of video images using a first set of multiple stationary 
overhead cameras, not capable of pan, tilt or zoom movement, sufficient to form a 
contiguous field-of-view of said entire performance area, and where each said first set 
camera provides images continuously throughout said entire time duration ; 

simultaneously analyzing said first stream of video images from said first set of 
multiple stationary overhead cameras in order to continuously detect the presence of 
any one or more participants or objects within each and every camera's view, to 
determine each detected participant's or object's relative two-dimensional centroid 
coordinates within that view, and to use this determined information from all said first 
set cameras for updating a real-time tracking database of the two-dimensional centroid 
location of each participant or object , relative to said entire performance area, where 
said real-time tracking database includes all current and past said centroid locations 
determined throughout said entire time of said event: 

using said determined participant and object locations stored in said real-time tracking 
database to automatically and individually adjust without the aid of an operator, the 
current view of each camera in a second set of multiple stationary cameras , capable of 
pan, tilt and zoom movement distinct from said first set of stationary overhead 
cameras, and 

capturing a second stream of video images for viewing and recording from said 
second set of multiple stationary cameras, wherein said second stream of video 
images create independent views of one or more of the participants or objects within 
said entire performance area and wherein said second stream of video images is not 
used to either determine any participant's or object's two-dimensional centroid 
locations or to otherwise update said real-time tracking database. 
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Claim 1 68. (Amended) The method of claim 1 67 wherein the contiguous field-of-view formed by 
said first set of stationary overhead cameras is substantially parallel to the ground surface 
within said entire performance area. 

Claim 1 69. (Amended) The method of claim 1 67 for further providing a three dimensional model of 
the movements of the participants or objects, comprising the additional step of: 

analyzing said real-time tracking database and said first and said second streams of 
video images to determine the ongoing relative three-dimensional coordinates of one 
or more specific, non-centroid locations on each participant or object and for updating 
said real-time tracking database to include the additional three-dimensional 
coordinates of all detected locations matched with the respective participants or 
objects. 

Claim 1 70. (Amended) The method of claim 1 69 further comprising the steps of: 

adhering markers onto the one or more locations on each participant or object to be 
tracked; 

detecting said markers using computer based image analysis of said first or said 
second streams of video in order to determine the ongoing relative three-dimensional 
coordinates of each marker, and 

updating said real-time tracking database to indicate the ongoing relative three- 
dimensional coordinates of said markers matched with the respective participants or 
objects. 

Claim 171. (Amended) The method of claim 1 70 wherein said markers are substantially transparent 
to the participants, further comprising the steps of: 

using said markers that reflect, retroreflect or fluoresce primarily non-visible energy 
and are therefore substantially visibly transparent, and 

using one or more energy sources to emit non-visible energy throughout said entire 
performance area, to be reflected or retrore fleeted off of said markers, or emitting 
energy that is fluoresced in the non-visible spectrum by said markers, where the non- 
visible energy is detectable by said first or said second set of stationary cameras. 
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Claim 1 72. (Amended) The method of claim 1 71 wherein the participants are additionally identified , 
further comprising the steps of: 

placing at least one uniquely encoded marker onto an upp er surface of each participant 
to be identified; 

detecting and recognizing each encoded marker using computer based image analysis 
of said first stream of video images , and 

updating said real-time tracking database to indicate the identity of each participant 
matched to their coordinates. 

Claim 1 73 . (Amended) The method of claim 1 72 wherein the uniquely encoded markers are 
substantially transparent to the participants, further comprising the steps of: 

using said uniquely encoded markers that reflect, retroreflect or fluoresce primarily 
non-visible energy and are therefore substantially visibly transparent, and 

using one or more energy sources to emit non -visible energy throughout said entire 
performance area, to be reflected or netroreflected off of said encoded markers, or 
emitting energy that is fluoresced in the non-visible spectrum by said encoded 
markers, where the non-visible energy is detectable by said first set of stationary 
overhead cameras. 

Claim 1 74. (Amended) The method of claim 1 67 wherein the participants are additionally identified , 
further comprising the steps of: 

placing at least one uniquely encoded marker onto an upp er surface of each participant 
to be identified; 

detecting and recognizing each said encoded marker using computer based image 
analysis of said first stream of video images , and 

updating said real-time tracking database to indicate the identity of each participant 
matched to their coordinates. 

Claim 1 75. (Amended) The method of claim 1 74 wherein said uniquely encoded markers are 
substantially transparent to the participants, further comprising the steps of: 
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using uniquely encoded markers that reflect, retroreflect or fluoresce primarily non- 
visible energy and are therefore substantially visibly transparent, and 

using one or more energy sources to emit non-visible energy throughout said entire 
performance area, to be reflected or relroreflected off of said encoded markers, or 
emitting energy that is fluoresced in the non-visible spectrum by said encoded 
markers, where the non-visible energy is detectable by said first set of stationary 
overhead cameras. 

Claim 1 76. (Amended) The method of claim 1 67 for videoing the movements of two or more 
participants from optimal viewpoints, further comprising the steps of: 

using two or more cameras in said second set of stationary cameras, and 

during the step of individually adjasting the pan, tilt or zoom movements of each 
camera in said second set of stationary camera s, capable of pan, tilt and zoom 
movement, dynamically considering the location of each participant or object with 
respect to the view of each said second set camera and automatically reassigning any 
one or more said second set cameras currently following any one or more participants 
or objects to instead follow a different one or more participants or objects based upon 
which camera views are best situated to capture the programrnatically desired view of 
any given participant or game object. 
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